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ABSTRACT 

The Further Education Unit's (FEU's) current quality 
projects in Great Britain include Trialling TQM [total quality 
management] , Maintaining Quality During Curriculum Change, and 
value-added work. The FEU value-added project consisted of a survey 
of 125 technical colleges to discover the extent of value-added 
activity and a short-term working group set up in response to the 
survey to develop the value-added work in further education. The 
survey demonstrated that there is widespread interest in the 
development of value-added methods and techniques and that, despite 
some interesting small-scale practice, few colleges had developed 
systematic ways of capturing the data needed to measure value added. 
The short-term working group of representatives from five colleges is 
continuing to work on defining, measuring, and reporting value added. 
Before undertaking the development work in their own colleges, 
members have needed to develop a shared understanding of the 
measurement of value added and the additional difficulties in 
applying such measures to the further education context,- Measuring 
value added involves a longitudinal analysis of many individual 
student performances. Measuring value in further education is 
problematic because students on entry often have more diverse 
attainments than 16-y3ar-olds and the curriculum includes many 
different vocational levels. A need clearly exists for resources and 
staff expertise to undertake development of value-added work in 
further education. (YLB) 
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VALUE ADDED IN 
FURTHER EDUCATION 

One aspect of FEU^s continuing quality work 



Since the piihlicatioti of Quality Matters (FEU 1991), FEU has 
continued work on several lon^-term quality proiects. This 
bulletin: 

• provides an update of FEU's current quality work 
0 looks in detail at one of those proiects 
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FEU'S QUALITY WORK: AN UPDATE 

In addition to its value-added work, FEU's current quality 
projects include: 

• TriallingTQM (RP671) 

This project has been following two colleges as they have 
introduced TQM. It will report in the winter of 1993-4. 

• Maintaining Quality During Curriculum Change 
(RP 742) 

This project has identified and is trialling a three-part 
instrument to monitor quality during periods of rapid 
curriculum change. It is also identifying good practice 
which will help course leaders and college curriculum 
managers maintain quality during the process of 
curriculum development. It will report in the spring of 
1994. 

Future publications, which will draw on a' variety of 
recently completed development work will focus on: 

• the use of ser\'ice standards in FE; 

• the integration within colleges of a variety of quality 
initiatives such as quality systems, Investors in People, 
curriculum review and evaluation, staff appraisal and 
human resource management (HRM). 

Recent market research has demonstrated that, in addition 
to the above, there is an expressed need within colleges for: 

• the development of a common quality framework; 

• guidance on how to demonstrate to funding bodies and 
other agencies that quality systems are operating 
effectively and resulting in continuing improvement. 

FEU is considering how best to continue to respond to 
these needs. 
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VALUE ADDED IN FE 



This bulletin: 

• explores the uses of value added in FE; 

• points to some ways forward; 

• identifies some issues still to be resolved; 

• shows how FEU will develop the work further. 



What is value added? 

The measurement of value added entails comparison 
between the characteristics and attainments of learners at 
entr>' (the input data) and their achievements at exit (the 
output data). 

Through the aggregation of such measurements institutional 
and national norms can be computed. Comparisons can 
then be made between individual distance travelled and the 
outcomes predicted by these norms. 



Why is there so much interest in value added in 

FE? 

Current interest is high partly because colleges are being 
encouraged by external bodies, such as the Funding 
Councils, to develop internal quality assurance mechanisms 
(e.g. FEFC 93/11 and 93/28), but also because such 
measures may eventually provide ways of: 

• enhancing objectively-based guidance to students at 
entry; 

• motivating students on programme by setting realistic 
targets; 

• monitoring internally the effectiveness of various parts of 
the college; 

• assessing internally the achievements of a college and 
identifying areas for future action; 

• expressing individual, section or institutional 
achievement with greater validity than raw examination 
or qualifications data; 

• accounting for the use of public money. 

In addition work initially undertaken in schools by the 
Audit Commission (1991) and the A-level Informative.. 
System (ALIS) work (see below), have been extended with 
some success to the FE sector, and hopes are high that 
similar ways might be found to measure the value added by 
voc.aonal programmes as well as A-level courses. 
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A-level Information System 

A LIS is based in the Curriculum, Evaluation and 
Management centre of Newcastle University and has been 
running for some ten years. 

• It has provided value-added information for schools and 
more recently colleges, initially in the North East and 
nov^ across the country. 

• The analysis is based upon the work of individual 
departments and does not set out to give a global picture 
of school or college performance. It is argued that 
subject-by-subject analysis can give more useful and 
practical information for student guidance and staff 
development than any aggregate measure. 

• The system depends on up-to-date analysis of individual 
A-level trends and performance year by year over large 
samples. 

• In addition to the school or college examination results, 
ALIS also provides feedback data on student attitudes 
and on institutional characteristics in relation to teaching 
and learning policies. 

• The ALIS approach is also being extended to a study of 
lull-time vocational courses. 

• ALIS is an example of how detailed and authoritative 
value-added information and analysis can be provided 
to schools or colleges from an external centre. The 
contract colleges make with ALIS determines in what 
form such information is delivered and will not provide 
the immediate responsiveness and accessibility of an in- 
house system. Schools and colleges need to have 
systems which capture and provide input and output 
data for ALIS. 



FEU'S VALUE-ADDED PROJECT 



The FEU project consisted of: 

• a survey of 125 colleges to discover the extent ol value- 
added activity m early 1992; 

• a short-term working group sot up in response to the 
survey to develop further the value-added work in FE. in 
relation to A levels, vocational programmes and non- 
traditional entry. 

The survey demonstrated that: 

• there was widespread interest in the development of 
value-added methods and techniques; 

• despite some interesting small-scale practice, few 
colleges had developed systematic ways of capturing the 
data needed to measure value added. Colleges which 
had subscribed to the ALIS project were sometimes 
exceptions. 

The short-term working group was set up with 
representatives from five colleges. Us remit was to work 
further on defining, measuring and reporting value added in 
further education. It was supported by a steering committee 
with specialists actively involved in this lield. These 
included staJ! irom Greenhead Sixth-Form College, the 
National Foundation for Educational Research (NFER), the 



Business and Technical Education Council (BTEC), the 
Schools Examination and Assessment Council (SEAC) and 
ALIS as well as other FE college representatives. 

Before undertaking the development work in their own 
colleges, members needed to develop a shared 
understanding ot: 

• the measurement of value added; 

• the additional difficulties in applying such measures to 
the FE context. 

Measuring value added 

Measuring value added involves a longitudinal analysis of 
many individual student performance.;. Confidence in the 
\'alidity of measurements may be increased by using larger 
samples, for example by adding data from successive 
cohorts of students. Input and output data need to be agreed, 
captured and coded, usually via a standardised points 
system. 

EXAiMPLE: The Audit Commission (1991 ) used the 
following for GCSE and A level scores: 

INPUT DATA 

GCSE grade A B C D E F G U 
Points awarded 7 h 5 4 3 2 1 0 

OUTPUT DATA 

A-level grade A B C D E O U 
Points awarded 5 4 3 2 1 0 0 
(based on the UCCA points system) 

A statistically significant correlation must be established 
between input data and output data. 

EXAiMPLE: The Audit Commissions data on 1271 voung 
people who took A levels in the summer of 1988 showed 
there was a significant correlation between GCSE scores 
and A-level results: that is the higher the GCSE points at 
entry, the higher the A-level scores are likely to be. 

For ease of understanding, the whole range of individual 
scores can be plotted on a graph and then, using a statistical 
technique known as regression analysis, the national norm 
can be represented as a straight line. Once this has been 
done, subsequent individual or group scores can be plotted 
and compared with the norm. 

The concept of value added 
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More sophisticated statistical analyses, such as multi-level 
modelling techniques, which differentiate between such 
factors as gender or age in the measurement of value added, 
can also be employed. They can demonstrate whether, other 
things being equal, any overall value added is evenly 
distributed within a college or whether, for instance, it 
contributes more to older or male students (and thus less to 
younger or female students). 

The analyses resulting from such methods do seem to have 
the potential to meet many of the requirements of the FE 
system; for example, to have quality measures that 
demonstrate FE's effectiveness to outside agencies and 
enhance internal quality management. There seems 
therefore to be a strong case for pursuing further the use of 
value added measurement in FE. 

So far the examples have related to the GCSE and A-level 
results of young people in schools, and apart from so:"^.e 
technical concerns about the comparability of various 
subjects and examination boards, such value-added 
techniques have gained considerable credibility over recent 
years. However once the move into further education is 
made, additional difficulties are encountered. 

Value-added measures in FE 

Measuring value added in FE is more problematic because: 

• FE students on entr)- often have more diverse 
attainments than 16 ear olds; 

• the FE curriculum includes many different vocational 
programmes as well as A levels. 

The implications of this for measuring value added are: 

Student diversity 

GCSE scores may not be valid input data for FE students 
whose attainment levels on entry may have more to do with 
life experiences than GCSE or 0-level results, many years 
previously. For example, the starting points in one college 
programme reviewed by the project varied from graduates 
to people with no formal qualifications at all. Thus there will 
probably be a need to find alternative input data for many 
FE students. 

Vocational programmes 

The correlation between GCSE scores and the successful 
achievement of vocational qualifications is much weaker 
than that between GCSEs and A levels. Given that GCSEs 
and vocational qualifications are assessing different sorts of 
learning this should not be surprising. The closer the 
learning requirements of a vocational programme are to the 
sort of learning required for A levels, the greater the 
correlation with GCSE scores. Thus for example BTEC 
National Diplomas in Business Studies or Science have 
greater correlation with GCSE scores than those for Leisure 
Studies or the Performing Arts. Again this means that 
alternative input data will be required to measure value 
added. 



The grades awarded for achievement on vocational 
prog^-'^mmes may be less finely differentiated than GCSEs 
and A levels, hor example, in relation to NVQs, students are 
assessed as simply competent or not. This will limit the 
extent to which the distance travelled can be calibrated and 
finely analysed. This problem is exacerbated if, in addition, 
students coming onto a programme have very similar GCSE 
grades. In this case both the input and output data would 
lack differentiation. 

Many vocational programmes do not have a single form of 
accreditation such as A levels. For example, many 
programmes are assessed on a unit basis. This necessarily 
complicates the simple scoring system used for input-output 
measurement with GCSEs and A levels. 

Despite these technical difficulties, but with an 
understanding of them, colleges in the project undertook 
short-term, often retrospective, analyses of data within their 
own institutions, in an attempt to develop the use of value 
added techniques in FE further. For example: 

College A with a large and varied portfolio of GCSE and 
A-level piogrammes, investigated the characteristics of 
its mainly adult student groups, for whom value added 
cannot be measured using an input score based on GCSE 
results alone. A questionnaire was used to obtain 
information about such factors as motivation, prior 
qualification, general academic attainment, period of 
absence from formal education, and age. 

As sources of data, the college also drew on its 
information system, paper-based departmental student 
records, and computerised records of student 
attendance held on a database. 

The college's catchment area is also served by a sixth- 
form college and the prospect of public comparison of 
results was an incentive to the college to find ways of 
identifying and highlighting the strengths of its 
programme. 

From the college's experience it became clear that a vast 
amount of work would need to be done to achieve valid 
and reliable measurements of value added for a very 
heterogeneous student population. While the 
complexity of the task was convincingly demonstrated, 
the results in terms of measurement of value added were 
meagre. 

This college, outside the project itself, also used value- 
added techniques to test the effectiveness of new 
methods of teaching spelling to students on vocational 
courses. Both sample and control groups were tested, 
and statistical analysis of the final test scores enabled the 
tutors involved to distinguish between the learning gain 
attributable to the new teaching methods and the level of 
performance predicted by initial testing. 



College B is a large tertiary college with major regional 
and some national tunctions. It has a full-time 
equivalent (FTE) student roll of 5000 with 400 FTE 
teaching staff. 

One of the vice-principals has developed his own interest 
in value-added measurement by creating a dedicated 
database applying originally to full-time general 
education students, and recently extended to full-time 
vocational students. 

While the college's main management information 
system (MIS), Fretwell-Downing, supports the value- 
added project through its Examinations Entr\' module, 
the main analysis is carried out on an Acorn Archimedes 
micro-computer, using software refined over several 
years. 

The recent extension of work to vocational areas means 
that the kinds of analysis carried out for A-level students 
can now be used for many BTEC National Diploma 
programmes. 

The database permits the following analyses: guidance 
and placement work with individual students: target- 
setting and development work with departmental and 
programme staff; trend-spotting in o\'erall college 
performance; analysis ot performance by age, gender, 
prior attainment and feeder mstitution. 

The location of the system (with its expertise) in the 
senior management of the college ensures that its 
potential is recognised in terms of overall college 
development. However, the capacity of the main MIS to 
include the value-added database would certainly 
enlc. its use and influence. 

During the lifetime of the project FEU became aware of a 
number of other colleges developing value added 
techniques. These include: 

College C, a small college ser\'ing a mainly rural 
catchment area, which carried out a project funded by 
the National Council for Educational Technology 
(NCET) to extend the college's computerised MIS to 
monitor student progress in terms of value added. The 
facilities provided by EMIS have been extended, and 
additional information taken from student record cards 
has been assimilated. A scoring system covering the 
relevant range of academic and vocational examination 
results has been applied to obtain input and output data. 
Tlie college sees the benetit to staff in terms of being able 
to offer predictions of student performance, identify 
needs for specific tutorial support, and easily provide 
information for external agencies. The students 
benefit from better targeted guidance and support. 

College D, a college with a well established track record 
of innovation, has been trialling the use of standardised 
psychometric tests for initial diagnostic assessment of 
non-standard applicants for BTEC National programmes. 
Test results have boon used as a basis for initial guidance 
and the design of on-course learning support. The 
assessment team has compared the performance of 
college students with national norms, and has generated 
local norms from the data collected. The predictive 
validity of the assessments has been reviewed by 
comparing test results with subsequent performance on 
relevant BTEC modules. As a resulL the learn has 



recommended that a variety of appropriate forms of 
aptitude, attainment and diagnostic testing should be 
adopted to assist selection and guidance. The aptitude 
tests used in the trial could be applied to best advantage 
as a tool for screening students likely to need additional 
learning support. 



GENERAL FINDINGS 



There is a need to develop in colleges a shared 
understanding of the concepts and methods associated 
with value added, and the uses to which such measures 
can be put, 

Many colleges are unable to access the necessary data for 
value-added calculations from their CMIS. 

Statistical data can be successfully analysed for value- 
ad Jed purposes using commercially available statistical 
software packages and databases. 

Earlier research, indicating that the standard value- 
added method of relating tinal outcome to starting 
qualification can be usefully applied in FE, was 
confirmed. 

The Audit Commission's finding, that GCSE scores do 
not generally provide valid input data for measuring 
value added on vocational programmes, was confirmed. 

It is difficult to find a single methodology for recording 
input data for adults which would be equivalent to 
GCSE scores for younger students. 



UNRESOLVED ISSUES 



The experience of the project colleges and others shows 
clearly that there is a need for further work to: 

• ensure that college MIS make provision for the 

requirement to gain easy access to student input and 
output data; 

Recent evidence from FEU projects has shown that many 
colleges consider their MIS inadequate to meeting the 
developing demands placed upon them by external 
agencies and by the need to manage an increasingly 
flexible curriculum. 

This project confirmed this view in relation to measuring 
value added. Unless data on student characteristics and 
attainments are at the heart of college information 
systems, value-added systems will continue to be 
idiosyncratic, free-standing and dependent on the 
expertise and ingenuity of individual staff. 

This issue is explored in more detail in the recent 
bulletin, Management Information Systems and the 
Curriculum (FEU, 1993). FEU will continue to co-operate 
with the National CMIS Board, The National Association 
for Information Technology in Further Education 
(NAFTFE) and NCET in its work on this subject. 



ensure that staff generally are aware of the potential 
value of student records in the measurement of value 
added and thus of the importance of recording 
information systematically and keeping the records in a 
safe yet accessible place; 
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• explore and attempt to validate a wider range ot 
predictive and diagnostic tests as substitutes tor GCSE 
uiput data tor adults and in relation to many vocational 
programmes: 

• examine the implications ot the restricted grading range 
ot GN'VQs tor the measurement ot value added; 

• demonstrate convincingly to colleges the ways in which 
the technical difficulties associated with the 
measurement of value added in FE can be o\'ercome; 

• show how the measurement and analysis of value-added 
data can help colleges give a better service to students 
and enable them to manage quality internally as well as 
meet the requirements of external agencies. 

Despite these unresolved issues and indeed the difficulty in 
finding the resources and staff expertise required to 
undertake this work, colleges already working in this area 
are doing so because they believe value-added analyses can 
help them deliver: 

• increased student satisfaction: 

• better retention rates: 

• higher productivity; 

• enhanced standing with funding agencies. 

They also seem to have recognised the need within colleges 
for: 

• senior managers with an understanding both of the 
potential of value added for guidance, curriculum 
management and public accountabilitv purposes and of 
the need for statistical rigour; 

• ^statistical expertise; 

• access to expertise in information systems design and 
programming; 

• >taff development to raise awareness of the requirements 
and applications of measuring value added. 

1-EU will contmue to support the development of \ alue- 
added work in FE, with phase 2 of this project starting in the 
autumn ot 19^)3. During the academic years 1^)^3-5, FEU will 
work clo'^eiy with five colleges, trying to find ways to 
overcome some of the technical difficulties involved in 
measuring value added with a diverse student population 
on a variety of academic and vocational programmes. 
Advanced GWQs (level 3) will provide the focus for much 
ot the work. 
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FEU welcomes views on the ideas expressed in this bulletin. 
Please contact: 

Dr Stella Dixon 

FEU, South West Regional Office 

Coombe Lodge 

Blagdon 

BRISTOL 

BS18 6RG 

Telephone no: (0761) 462503 
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